Various techniques can be used to produce three-dimensional (3D) images with a variety of lighting conditions and textures. For example, in a structured-light assembly, a pattern is projected onto a subject, an image of the pattern is obtained, the projected pattern is compared to the collected pattern, and differences between the two patterns are correlated with depth information. In other words, distortions in the pattern are correlated with depth. This technique can be useful for low-light and low-texture objects or scenes. In structured-light assemblies, the projected pattern often uses infra-red (IR) radiation. However, projected IR techniques cannot readily be used when ambient IR is high (e.g., outdoors). Further, the projected IR pattern typically cannot be projected over long distances, and the accuracy of the system tends to be highly sensitive to the accuracy of the projected pattern. This situation can arise, for example, when part of the pattern is occluded by a contour or feature of the object or scene being imaged.
Stereoscopic image capture is another technique for producing 3D depth information. Stereoscopic image capture requires two imagers spatially separated by a baseline distance (B). Depth information (Z) can be extracted from the measured disparity (D) between matched pixels in each image, and the focal length (F) of the imagers according to: Z=(F×B)/D. Distinct features and/or intensity of the object or scene, such as texture, are required to facilitate matching pixels in both images. Consequently, stereoscopic pixel matching is particularly challenging for objects or scenes with low texture, or low-light scenes (where the observable features and intensity of textural differences are diminished). Often an object or scene may possess either high- and low-texture or high and low-light regions, where combinations of both stereoscopic and structured-light image capture are useful for generating depth information of the entire scene.
Structured-stereo (also referred to as active stereo) combines the benefits of both structured-light and stereoscopic image capture. In structured-stereo assemblies, a pattern is projected onto a subject, and images of the pattern are collected by two spatially separated imagers (e.g., IR imagers). Pattern capture by two (or more) imagers can eliminate the problems of pattern occlusion. Moreover, in low-texture and low-light scenes the projected pattern simulates the essential scene texture required for stereoscopic matching of pixels. The disparity between pixels is then correlated with depth. In some cases, disparities between collected patterns may be correlated with depth.
In addition, more advanced structured-stereo embodiments combine image projection (e.g., IR) and pattern-facilitated stereoscopic matching with additional stereoscopic image capture (e.g., RGB). Such an implementation is useful to gather depth information of a scene that is characterized by both low and high texture and/or low and high lighting features. Both structured-stereo and additional stereoscopic image capture are employed to correct the deficiencies of each method. Specifically, depth information from low-texture/low-light regions of a scene is combined with depth information from high-texture/high-light regions of a scene, where each is gathered by structured-stereo (e.g., IR) and additional stereoscopic image capture (e.g., RGB), respectively.
Some imaging assemblies that employ structured-light combinations or structured-stereo project an IR pattern and collect images with an IR/RGB color filter array (CFA), where RGB refers to red (R), green (G) and blue (B) light in the visible part of the spectrum, and where the IR and RGB sensitive regions of the imager are contiguous (i.e., part of the same mosaic or CFA). An imaging assembly that uses such a combined IR/RGB CFA can present various issues. First, crosstalk from the IR-pixels to RGB pixels of the imager can be significant, which can lead to images of reduced quality. Further, the combined IR/RGB CFA generally precludes incorporation of dedicated (e.g., wave-length-specific) optical elements such as lenses designed specifically for IR, red, green or blue light. Consequently, aberrations such as chromatic aberrations can be produced. In order to collect IR radiation, combined IR/RGB imagers typically do not have IR-cut filters. Consequently, RGB pixels may sense a lot of IR radiation, resulting in significant noise.
Another issue that can arise in spatially separated imaging imagers such as structured-stereo assemblies is misalignment. Misalignment is particularly detrimental to the accuracy of the collected depth information. Calibration of the imaging assembly typically occurs before reaching the end-user (e.g., before the assembly leaves the manufacturing factory). During normal use, however, the user may drop the imaging assembly, causing some of the imagers to become misaligned.